Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ignore_permission_denied #80

Closed
wants to merge 2 commits into from
Closed

ignore_permission_denied #80

wants to merge 2 commits into from

Conversation

earshinov
Copy link

There is no reason to stop Django process and exit with error when watch encounters an incident such as an unreadable file, "permission denied" etc. So it seems reasonable to me to always pass ignore_permission_denied=True to watchfiles.

See also samuelcolvin/watchfiles#224, where this flag was introduced.

@earshinov earshinov force-pushed the ignore-permission-denied branch 2 times, most recently from 41f65bf to 3eb321c Compare March 28, 2024 09:17
@earshinov earshinov mentioned this pull request Mar 28, 2024
Closed
@earshinov
Copy link
Author

earshinov commented Mar 28, 2024 •
edited
Loading

Unfortunately, simply passing ignore_permission_denied was not enough to ignore errors in our use case, so I

  • enhanced this PR with a try..except to catch exceptions, log and ignore those
  • filed an issue to watchfiles so that errors are ignored on their side

You can find a full description of the problem we are facing in the watchfiles issue.

With this try..except, I see exceptions finally being ignored (instead of crashing our Dajngo app):

[WARNING] [2024-03-28 09:49:58] [/usr/local/lib/python3.10/site-packages/django_watchfiles/__init__.py::watch_safely (line 43)]: error in underlying watcher: IO error for operation on /opt/project/.mypy_cache/3.10/grt_helpers/django/validators/__init__.data.json.185fc872ae08f752: No such file or directory (os error 2) about ["/opt/project/.mypy_cache/3.10/grt_helpers/django/validators/__init__.data.json.185fc872ae08f752"]
  Traceback (most recent call last):
    File "/usr/local/lib/python3.10/site-packages/django_watchfiles/__init__.py", line 37, in watch_safely
      yield from f()
    File "/usr/local/lib/python3.10/site-packages/watchfiles/main.py", line 122, in watch
      raw_changes = watcher.watch(debounce, step, rust_timeout, stop_event)
  _rust_notify.WatchfilesRustInternalError: error in underlying watcher: IO error for operation on /opt/project/.mypy_cache/3.10/grt_helpers/django/validators/__init__.data.json.185fc872ae08f752: No such file or directory (os error 2) about ["/opt/project/.mypy_cache/3.10/grt_helpers/django/validators/__init__.data.json.185fc872ae08f752"]
[WARNING] [2024-03-28 09:50:25] [/usr/local/lib/python3.10/site-packages/django_watchfiles/__init__.py::watch_safely (line 43)]: error in underlying watcher: IO error for operation on /opt/project/.mypy_cache/3.10/sentry_sdk/session.data.json.9ab62077a5ffd0bb: No such file or directory (os error 2) about ["/opt/project/.mypy_cache/3.10/sentry_sdk/session.data.json.9ab62077a5ffd0bb"]
  Traceback (most recent call last):
    File "/usr/local/lib/python3.10/site-packages/django_watchfiles/__init__.py", line 37, in watch_safely
      yield from f()
    File "/usr/local/lib/python3.10/site-packages/watchfiles/main.py", line 122, in watch
      raw_changes = watcher.watch(debounce, step, rust_timeout, stop_event)
  _rust_notify.WatchfilesRustInternalError: error in underlying watcher: IO error for operation on /opt/project/.mypy_cache/3.10/sentry_sdk/session.data.json.9ab62077a5ffd0bb: No such file or directory (os error 2) about ["/opt/project/.mypy_cache/3.10/sentry_sdk/session.data.json.9ab62077a5ffd0bb"]

@earshinov
Copy link
Author

All references at a glance:

@earshinov earshinov force-pushed the ignore-permission-denied branch from 3eb321c to ed9ac16 Compare March 28, 2024 10:03
@earshinov earshinov mentioned this pull request Mar 28, 2024
Open
@earshinov earshinov force-pushed the ignore-permission-denied branch from ed9ac16 to 0a57fc7 Compare March 28, 2024 10:07
@adamchainz
Copy link
Owner

You wrote on the watchfiles issue:

For the moment, I am not ready to tinker with Rust, so I made a PR into django-watchfiles to circumvent this on the outside. But really, I think that this code should be unnecessary and the errors should be ignored right in watchfiles.

Let’s wait and see what Samuel replies.

@earshinov
Copy link
Author

@adamchainz , Hey Adam! There's been no reply in my PR to watchfiles for a month. Do you think you could now consider merging this PR into django-watchfiles? If my PR is merged into watchfiles, I can volunteer to update django-watchfiles again to remove the workarounds which we add here in this PR.

@adamchainz
Copy link
Owner

Samuel has been busy launching Pydantic Logfire: https://pydantic.dev/ . I would still wait for a reply there.

Until then you can install your fork, if you're not already: https://adamj.eu/tech/2019/03/11/pip-install-from-a-git-repository/

I would rather not merge any workarounds here. django-watchfiles is still alpha and I'd rather only add things we're sure we need, to iterate towards a 1.0.0 release.

@adamchainz
Copy link
Owner

If I understand correctly from these comments: notify-rs/notify#581 (comment)

Then your issue has been fixed with watchfiles 0.24.0.

I am going to close this for now, then.

Adding ignore_permission_denied sounds like it might help someone, so I may revive that change when there's a relevant report. But I'm wary about just blanket ignoring such errors, because they may also indicate a project fault.

Thank you for providing enough links to chase up on!

@adamchainz adamchainz closed this Oct 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@earshinov @adamchainz